Providing an ability to store large amounts of data (or records), which data can be quickly accessed, modified, and re-stored, is typically a mandatory requirement for large, intermediate and even small computing systems. These computing systems, for example, take the form of a host processor, such as an IBM System/360 or IBM System/370 processor for computing and manipulating data, attached to an IBM 3990 storage controller, which is further connected to a group of direct access storage devices (DASDs) such as IBM 3380 or 3390 DASDs. While the host processor provides substantial computing power, the storage controller provides the necessary functions to efficiently transfer, stage/destage, convert and generally access large databases. These databases, as stored on DASDs, for example, can easily exceed several gigabytes of data.
DASD storage, for example, magnetic or optical disks, stores bits of data as micrometer sized magnetic or optical altered spots on a disk surface for representing the "ones" and "zeros" that make up those bits of the data. Magnetic DASD, includes one or more disks that are coated with remnant magnetic material. The disks are rotatably mounted within a protected environment. Each disk is divided into many concentric tracks, or closely spaced circles. The data is stored serially, bit by bit, along each track. An access mechanism, known as a head disk assembly (HDA), typically includes one or more read/write heads, and is provided in each DASD for moving across the tracks to transfer the data to and from the surface of the disks as the disks are rotated past the read/write heads.
Data or records are generally stored in DASD in count-key-data (CKD) format or in fixed-block architecture (FBA) format. CKD record storage is desirable because the records are stored efficiently, that is, wasted disk space is minimized. Records stored in the CKD format include a count area, a key area, and a data area. The count area is typically an n-byte field that identifies the record by describing the physical address (track number), an identifier (including a cylinder number, a head number, and a record number), a record format, and the length of the record (how many bytes of data make up the record, and a key length (how many bytes make up the following key area). The key area can be many bytes long, for example from one to 256 bytes, for uniquely identifying the record. The key could be a part number, employee identification number, an account number or other useful identifier. The data area is the actual data and can vary from a few bytes to millions of bytes. CKD records, in addition to efficient storage, conveniently provide an ability to find a record based upon the key. Also, because the CKD record length is defined and known, the CKD record can be written contiguously, with another record written at the end of the previous record without wasting disk space between records. A drawback to the CKD record format is that an entire track may need to be searched to find the key for accessing the record.
Storing data in the FBA format requires using a DASD especially formatted for such storage. FBA record storage advantageously provides simpler processing and faster access at a reduced storage efficiency cost. Furthermore, FBA type DASDs are available at lower cost. DASDs having FBA record storage capability divide the tracks into many sectors, with each sector having a predetermined number of bytes. As an example, each track might be divided into seventy two sectors, with each sector capable of storing 512 bytes of data. Each sector provides both location or address identification and data storage. In the current example, if a record contains less than 512 bytes, then the area in the sector following the data is unused and typically padded (filled to the end with "zeros").
FBA type devices generally offer better performance than the CKD type devices. Additionally, FBA type devices are widely available offering small form factor packaging with storage capacities exceeding one gigabytes. Due to the proliferation of FBA devices, not surprisingly, users desire to store previously written CKD records in the FBA format. Writing a CKD record to an FBA device requires mapping the CKD record and sometimes converting the CKD record format. Several different conversion techniques are known for modifying such CKD records formats.
One method of converting CKD records into FBA records involves placing each area of the CKD record into separate sectors. Thus, the count field is written to a first sector, the key field is written to a second sector, and the data field is written to one or more additional sectors. The efficiency of this conversion format is dependent upon both the size of the records and the size of the sectors. The worst case scenario occurs with small records and large sectors, where substantial disk space is wasted in unused areas of the sectors in the count and key fields, as well as unused area in the data fields. On the other hand, this conversion may be quite efficient for cases of smaller sector sizes coupled with generally large data fields since the overhead of sectors for count and key fields is minimized. Writing CKD records to FBA devices in this manner is very fast since additional mapping computations are minimal.
Another method for providing CKD record to FBA record conversion involves packing the CKD records. In this conversion method, a first CKD record is written to the FBA device contiguously (the count, key and data areas are not separated by sector boundaries). The second CKD record is written starting where the first CKD record finished, which location could be in the middle of a sector. Writing CKD records in this manner provides excellent packing density. Larger sector sizes, however, tends to degrade system performance since data must be read starting from the beginning of a sector and much of that data may be of the record preceding the desired record.
Approaching the CKD record mapping problem from another perspective involves storing the CKD records in an unpacked format. In the unpacked format, system performance is excellent but packing density suffers. Each CKD record is written starting at a sector boundary and continues for as many sectors as needed. Hence, records can be found quickly without reading unwanted data. The space from the end of each record to the end of the last sector is padded and hence wasted. The wasted space can be undesirably large in cases of small records, and further exacerbated by larger sector sizes.
Yet another CKD record to FBA record format conversion involves storing the count and key areas of the CKD records in one area or table of the FBA device (and copying such table to electronic memory for improving read response time), and storing the data fields contiguously in the remaining sectors of the device. The tables in memory can then be used to more quickly determine the location of data field. Additionally, the data fields can be stored efficiently since they are stored contiguously. A drawback of the described conversion is that in cases of very large numbers of records, increasingly larger electronic memories for storing the tables are required.
Given that the size of data records often vary from one computer application to another (e.g., employee records may be relatively small and graphical image records may be relatively large), and the sector sizes from one FBA device to another can vary (e.g., from 512 bytes to 2048 bytes or more), and the number of records stored in a database vary substantially, mapping all types of records in a single format regardless of record or sector size does not provide optimum conversion for the different environments.
Accordingly it is desired to provide a method and storage system for fuzzy mapping CKD records to a FBA architecture such that storage space is efficiently used and system response time for staging such records is improved.